^z28th January 2026 at 9:54am
Gary Marcus's essay "Generative AI’s crippling and widespread failure to induce robust models of the world" (28 June 2025) is thoughtful, longish, semi-technical, and worth reading. In brief:
- cognitive “world models” are “persistent, stable, updatable (and ideally up-to-date) internal representations of some set of entities within some slice of the world” — like our mental models of where we left our shoes, what we shouldn’t ever say to our spouses, how big the Earth is, etc
- LLMs (mostly) don’t have “world models” — they just “extract correlations between bits of language”
- that’s why LLMs “hallucinate”, why they can’t play chess (or tic-tac-toe) reliably, and why they can’t generally learn
(cf Genius and Complexity (1999-05-25), Intelligence Augmentation (2001-08-25), Mind Children (2003-04-17), Taxonomy of Machine Learning (2017-02-02), ...) - ^z - 2026-01-28